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ABSTRACT 



This evaluation compared online (i.e., World Wide Web-based) 
and classroom instructional delivery methods for the Health 

Education/Physical Education course, "Fitness and Lifestyle Management, " at 
Brigham Young University (Utah) . The results of the study were intended to 
add to the discussion on the value of web-based courses as a means of 
providing quality instruction. A comparison of online and classroom students' 
achievement for specific lessons of the course show that the test scores were 
not significantly different. This indicates that either method for providing 
instruction could satisfactorily be used to learn course content when 
students fully participate and use the materials provided. A large majority 
of students who took online instruction indicated a definite opinion about 
which learning experience they preferred. Most students tended to prefer the 
classroom experience; however, students' reasons for choosing one method over 
the other had little to do with the quality of instruction. An individual's 
personal situation, social issues, and the perceived benefits of a specific 
delivery method were often the determining factor in a student's preference. 
It was clear that neither the classroom nor the online instruction fully met 
the learning needs of all students. The diverse needs of students warrant the 
existence of classroom courses with equivalent online courses. It was 
concluded that the university should continue to use and improve quality 
classroom instruction as they look to alternative methods for providing other 
quality learning experiences such as online instruction. (Contains 11 tables 
and 20 references.) (Author/AEF) 
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ABSTRACT 

This evaluation compared online and classroom instructional delivery methods for the 
Health Education/Physical Education course HEPE 129 - Fitness and Lifestyle Management. As 
our ability to provide instruction over the web is a relatively new phenomenon, the results of this 
study are intended to add to the discussion on the value of web based courses as a means of 
providing quality instruction. 

A comparison of online and classroom students' achievement for specific lessons of the 
course show that the test scores were not significantly different. This leads us to believe that 
either method for providing instruction could satisfactorily be used to learn the content of this 
course when students fully participate and use the materials provided. 

While test results were not significantly different, a large majority of students who took 
online instruction did indicate a definite opinion about which learning experience they preferred. 
Most students tended to prefer the classroom experience; however, the students' reasons for 
choosing one method over the other had little to do with the quality of the instruction provided. 

An individual's personal situation, social issues and the perceived benefits of a specific delivery 
method were often the determining factor in a student's preference. 

It was clear that neither the classroom nor the online instruction fully met the learning 
needs of all students. The diverse needs of students warrant the existence of classroom courses 
with equivalent online courses. We conclude that the University should continue to use and 
improve quality classroom instruction as they look to alternative methods for providing other 
quality learning experiences such as online instruction. 
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Evaluation Comparison of Online and Classroom Instruction for 
HEPE 129 - Fitness and Lifestyle Management Course 



INTRODUCTION 

The Health Education/Physical Education course HEPE 129 - Fitness and 
Lifestyle Management is a 2.0 credit course offered to meet the Wellness requirement for 
general education students at Brigham Young University (BYU). In conjunction with 
instructors currently teaching HEPE 129 on campus, the Independent Studies department 
at BYU has developed an online version of the course. The online course is accessed 
over the World Wide Web. This evaluation compares the two methods of delivering the 
instruction for the HEPE 129 course. As our ability to provide instruction over the web is 
a relatively new phenomenon, the results of this study are intended to add to the 
discussion on the value of web based courses as a means of providing quality instruction. 

Relevance of this Study 

The current enrollment of the HEPE 129 course for Fall and Winter semesters 
averages about one thousand students. With five or six sections of this course each 
semester, the class sizes often exceed two hundred students. The Health and Human 
Performance department at BYU has recently allowed BYU enrolled, degree-seeking 
students to take the Independent Study online course for credit. Both courses cover the 
same basic objectives in thirteen independent modules or lessons; both require students to 
complete similar additional learning activities; yet the methods for providing the 
instruction are quite different. 

There may be benefits and disadvantages to both delivery methods. This study 
attempts to determine the degree to which these two methods of delivering instruction for 
this course help students accomplish the objectives of the course. This is important for 
Independent Study because their mandate is to provide quality online courses. This is 
important for the department of Health and Human Performance because they hope to 
find that students taking the online course are being provided with a satisfactory learning 
experience. 

For the purposes of this evaluation, we will need to distinguish between these two 
methods of delivering instruction for this course. We will refer to the Independent Study 
version of this course as the Online Experience. The course as it is currently delivered on 
campus we will refer to as the Classroom Experience. 
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REVIEW OF RELEVANT LITERATURE 



Instruction and Learning 

A clear distinction needs to be made between learning and instruction. They are 
not the same things. Learning has been defined as the retention and understanding of 
knowledge and skills as well as the use of this knowledge and skill (Jonassen, 1988). 
Instruction is a planned learning experience. Students receive various types of instruction 
in hopes that they will learn. When students receive quality instruction it enhances the 
likelihood that learning will take place (Gagne, 1992). Quality instruction refers to both 
completeness in covering the desired learning objectives and usefulness to the students as 
a tool for accomplishing their learning. Improving the amount and quality of instruction 
can result in more effective and efficient learning (Walberg, 1994). It should also be 
noted that even when presented with quality instruction, it is possible that students will 
not learn. 

Our ability to determine precisely why students learn is often quite difficult in 
educational research as it is virtually impossible to control all the factors that affect 
learning (Gay, 1996). We can however evaluate the quality of the instruction provided 
based on quantitative and qualitative evidences available. This can be done for instruction 
delivered in the classroom or by the computer. 

Elements of Good Instruction 



Several studies have compared computer-assisted instruction with traditional 
instruction. Many of these studies have shown increases in learning when students use 
computer assisted instruction (Jonassen, 1988). However, using advanced technologies 
does not automatically guarantee quality instruction and learning (Zhao, 1998). This is 
the case whether technology is used as a supplementary activity in the classroom or as the 
sole method of instruction. The design and content quality of the instruction must be 
considered for both online and in-class instruction. 

While it is difficult to accurately predict whether learning will take place or to 
measure learning that has taken place, it is possible to identify elements of good 
instruction. Gagne, Briggs, & Wager (1992) indicate that the inclusion of certain 
educational elements in instruction will increase the chances that learning will occur. 

Introductory activities. Carefully designed introductory activities can have a 
great affect on students' motivation for learning. Such activities might include advance 
organizers or an introduction to the learning objectives. Any procedure that clearly 
organizes or outlines the learning that is to take place will influence what and how the 
student learns. Student’s attention and willingness to learn should increase when the 
physical and logical layout of the instructional material is improved (Harley, 1976). Not 
only do these organizational activities influence what students learn but also their attitude 
towards the material being taught. 



0 

ERIC 



2 



6 



Gaining attention and motivational elements . It is important to get students' 
attention and interest in the instruction being provided. Keller (1983) and Jones (1990) 
suggest that instruction should be designed to include creative, imaginative, and critical 
thinking activities along with relevant instruction as a means to improve student 
performance. Other motivational instructional strategies might include real life 
simulations, projects or apprenticeships (Reigeluth, 1996). In order for students to learn, 
they must find the instruction relevant and interesting. The inclusion of motivational 
elements will improve student satisfaction and desire for learning; it can be an effective 
way to increase learning (Dick, 1990). 

Review . Reminding learners of previously learned material can help students 
learn new material. This is especially important when the previously learned material is 
an essential prerequisite to the new material (Gagne, 1992). Dick and Carey (1990) 
indicate that the development of prerequisite and subordinate skills always facilitate 
learning. 

Clear presentation of information . Instructional information and material must be 
presented in a way that does not deter learning. The likelihood that learning will occur is 
diminished when the information to be learned is confusing or difficult to follow. The 
lesson information must also be appropriate for the target audience. Students tend to put 
less effort into their work when they perceive that a task required of them is too difficult 
(Weiner, 1992). In addition, expectancy theory suggests that when students believe their 
success or failure is attributed to outside influences, like inadequate instruction, they tend 
to put less effort into their work (Keller, 1983). One way to improve learning is to make 
sure that instructional information is presented in a clear manner. 

Discussion . Group discussions can be a powerful medium for learning (Collins, 
1994). Understanding increases when students are required to organize their thoughts and 
present them to others. Effective discussions can take place electronically online or 
verbally in small groups. Often this type of learning situation requires a moderator. 

Guided learning . Learning is more efficient when students are given direction. A 
guided exploration of the subject to be learned will improve the chances that students will 
learn the desired instructional outcomes. In addition, a report on the effectiveness of 
technology in schools (1997) indicates that many students learn more when they are 
allowed some control over the amount, review, and sequence of instruction. Most 
learning situations are more effective when the instruction guides the learners as needed. 

Eliciting performance and providing feedback . Another element of good 
instruction which Dick and Carey (1990) indicate always facilitates learning is practice 
with feedback. Providing feedback involves more than informing students what mark 
they received on a test. Unfortunately, tests are often only used to determine what has 
been learned, rarely are they used as a learning experience for students (Collins, 1994). 
Part of any course's instructional strategy should involve building skills using gradually 
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more difficult assignments and then emphasizing the importance of effort. All instruction 
should have a formative evaluation strategy that includes practice with feedback. 

The Human Factor. 



Research into courseware design and learner motivation has shown that motivated 
learners are often able to overcome the shortcomings of inadequate instruction (Keller, 
1988). These motivated learners utilize a variety of coping skills and exert persistent 
effort in order to achieve their learning goals. Bereiter and Scardamalia (1993) label such 
individuals intentional learners. Whatever the label, intentional learners end up learning 
despite the barriers of the learning environment and the inadequacies of the instruction. 

Current studies have found that most college students are not intentional learners. 
They are performing learners or conforming learners (Martinez, 1998). Performing 
learners are skilled learners who do what is needed to get the grade. They are task- 
oriented and rarely learn beyond what is required. They are usually able to overcome 
most of the imperfections of inadequate instruction. Performing learners achieve their 
best results when provided with quality instruction. 

Conforming learners are less skilled learners who require step-by-step instructions 
and are satisfied to simply complete assigned work. They expend little effort beyond the 
minimum requirements and have simple standards for themselves. Conforming learners 
are often adversely affected by poor instruction. 

Teaching in the Classroom. 

A common criticism of distance education from campus instructors is that it 
delivers information but does not develop understanding. It is a common opinion that 
students prefer in-class group instruction and that classroom instruction is more effective 
(Bates, 1994). However, the nature of the classroom experience dictates which elements 
of good instruction can be used effectively. The class size, time restrictions, classroom 
conditions and the lesson content often limit what can be done. These factors will affect 
the amount of learning that takes place. 

Advantages. In a classroom situation with a large class size, the most effective 
and most utilized method for delivering instruction is the lecture. A skilled lecturer uses 
many techniques to gain attention, motivate students, and direct the learning. He or she is 
able to review important prerequisite learning and might use a variety of pre-instructional 
activities to organize and introduce the learning. Experienced lecturers can also adjust the 
presentation of the instructional information to meet the immediate needs of the students. 

Disadvantages. Instruction in large groups goes contrary to mastery learning 
proposed by Bloom (1976). Group discussion is impractical, practice with personalized 
feedback is difficult to accomplish, and classroom conditions or the amount of 
information and the amount of time available sometimes adversely affect the instructor's 
presentation. 
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What is learned in the classroom depends heavily on the instructor. Not all 
instructors are good lecturers. Research indicates that student evaluations of courses do 
not accurately indicate the value and quality of the instruction as much as it does the 
competence of the instructor (Marsh, 1994). 

Technology and Teaching. 

Although computers have proven to be an effective method for delivering 
instruction, too often those who use the web as a method for delivering instruction simply 
extend what is already being done in the classroom. Quality online instruction must 
include elements of good instruction in such a way as to benefit from the potential 
advantages of the medium. In addition, the technology must be made as transparent as 
possible, it must take advantage of a variety of integrated teaching tools, and it should 
reduce data and help manage it (Zhao, 1998). If the online instruction is simply being 
implemented as a page-turner, a textbook may better serve the purpose. 

Advantages . Carefully designed online instruction can utilize a variety of 
techniques to gain attention, motivate students and direct the learning. Online instruction 
can provide review of important prerequisite learning and might use a variety of pre- 
instructional activities to organize and introduce the learning. 

Additional benefits of using computer assisted instruction include the ability to 
efficiently deliver multi-media information. Students can control the sequence and pace 
of the learning. Learning need not take place at a predetermined time and place. Quick 
access to alternate sources of information on the web can be provided; this information 
can be selected on the basis of an individual's needs and learning goals. Personalized 
learning can be accomplished with the use of expert systems. Group discussion is 
possible through instructional chat lines and bulletin boards. Immediate personalized 
feedback on assessment is possible. 

Disadvantages . Computer related hardware problems could occur. Poorly 
designed user interfaces may actually deter students from accessing the instruction; this 
would have a detrimental effect on learning. Information is sometimes difficult to read 
when appropriate information chunking techniques are not used. 

Changing Attitudes and Behavior 

An important aspect of learning involves students using knowledge and skills they 
acquire. Behavior is determined by one's attitude and this cannot be measured directly. 
How students use the knowledge and skills they have learned is an indication of attitude 
towards a specific topic. Attitude is measured by the direction and magnitude of student 
behavior. The purpose of instruction, as it relates to attitudes, is to make students aware. 
It is hoped that by making students aware of correct principles and social issues they will 
adapt their attitudes and behavior in a beneficial way. 
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Anyone can state a view or show understanding of a specific value then act 
against that view in a conflicting situation (Walberg, 1994). Conforming and performing 
learners often show evidence that learning has taken place only for the duration of a 
specific course. 



DESCRIPTION OF THE EVALUAND 



Classroom Experience 

The classroom experience is a traditional lecture driven model for giving 
instruction. Students are provided with a syllabus on the first day of class that provides 
an outline for the course. The course is divided into eighteen lectures that are referred to 
as modules. A different specialist instructor teaches each lecture. Students are tested on 
thirteen of these modules and must attend three of five elective modules. Students are not 
tested on the elective modules but an in-class quiz is given to record attendance. Six 
exams covering two or three modules each are administered through the B YU testing 
center. 



Students are required to complete several activities in addition to the module 
examinations. Students must participate in a physical fitness appraisal. This consists of a 
blood test and pre-test/post-test fitness appraisal at the beginning and end of the semester. 
Students also sign an exercise contract and complete an exercise log. 

There are two required textbooks for the course and students are given reading 
assignments for each lecture. Selected modules have reading material that can be 
obtained from the reserve library on campus or from the BYU Electronic Reserve System 
over the World Wide Web. 

Online Experience 

The content for the online experience was provided by various instructors of the 
HEPE 129 course and is intended to reflect the course taught on campus. Students access 
the Independent Study server and this course using the World Wide Web. There are 
thirteen lessons that students must take to complete the course. Students are provided 
with a set of four CD-ROM disks that contains video clips and animations intended to 
help teach the various lessons. 

Students are also required to participate in a physical fitness appraisal. This 
consists of an optional blood test and pre-test/post-test fitness appraisal at the beginning 
and end of the course. Students sign an exercise contract and complete an exercise log. 

All the material for the course is provided online. Students scroll through the 
materials provided and can access video clips, animations, sound clips, graphics and Web 
links that provide access to other online resources. In addition, the online experience 
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provides a “did you know” feature that gives interesting facts related to the lesson. The 
online experience features immediate feedback on lesson assignments and grade 
information as well as online processing of the fitness assessment forms. 

Identifying Differences 

Assessment. The online experience provides graded lesson tests called 
"Speedback" exercises. These tests provide immediate feedback to the student telling 
them how well they answered questions about material in the lesson. Speedback exercises 
are open book. These exercises can only be taken once and are worth sixty five percent of 
the final grade. The exercise contract, exercise log and fitness appraisals make up fifteen 
percent of the grade. There is an open book comprehensive final exam worth twenty 
percent of the final grade. The final examination is a proctored paper and pencil test. 

Students participating in the classroom experience take examinations at the 
testing center on campus. Each exam covers two or three modules and is a closed book 
test. Students can retake each exam twice for a fee of $3.25 each time. The retake 
examination is an alternate form of the test. Students can review the exam questions they 
missed prior to retaking an examination. Module exams are worth seventy percent of the 
final grade. Fifteen percent of the final grade comes from students attending a minimum 
of three elective modules. The exercise contract and exercise log makes up fifteen percent 
of the final grade. Students are also required to complete the fitness appraisal pre-test 
and post-test. 

Interaction with classmates . The online experience for this course has a bulletin 
board system that allows students to post messages to one another. Using the course 
discussion board is optional. Students are not required to use this feature to complete the 
course. The classroom experience allows for some in-class discussion. 

Optional modules . Students in the classroom experience are required to attend 
three of five elective modules. These modules include Marriage and Family Relations, 
Women’s Health, Environmental Health, Maintaining a Healthy Weight, and Crisis 
Intervention. The online experience has no elective module requirement. 

Fitness appraisal . Students participating in the classroom experience must sign up 
for an appointment to have the fitness appraisal completed. Fitness appraisals are 
completed outside of regular class time and are administered by trained staff. This 
appraisal requires that students give a blood sample for analysis. Students record the 
results of the fitness appraisal on a fitness questionnaire. The questionnaire is submitted 
and processed. Students receive the results approximately one week later. 

With the online experience, students complete the fitness appraisal on their own 
following instructions provided. They are not required to give a blood sample. The 
fitness results are recorded on a fitness questionnaire and processed immediately by the 
computer. 
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HEPE 129 Course Objectives 



The stated objective for the HEPE 129 course is to have students learn correct 
principles of health and physical fitness. Students will then apply these principles in 
designing and following a personalized fitness and lifestyle management program. 



EVALUATION QUESTIONS 

Several important and interesting questions emerged during initial discussion with 
stakeholders associated with this evaluation. Some questions cannot readily be answered 
given the conditions imposed on this study. This evaluation will focus on 

• The effectiveness of online and in-class instruction for teaching the principles of 
health and fitness as outlined in the HEPE 129 course. 

• The degree to which the instruction changes students’ attitudes about health and 
fitness. 

• Advantages and disadvantages of the two methods of instruction. 

The study will attempt to determine whether any significant difference can be 
found in test results and student feedback for the alternative methods of instruction used 
to provide information in this course. 

Student Performance Question. 

1 . Do students using different methods for delivering this instruction achieve similar 
results on assessment instruments currently being used in the on-campus course? 

Student Attitudes and Behavior Question. 

2. Do students feel their attitudes and behavior have changed after having taken this 
course? 

Value of the Instruction Questions. 



3. Do students use the instructional resources available? 

4. Do students prefer one delivery method more than the other? 

5. Do students find one method of instruction more useful than the other? 
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METHOD 



Participants 

Participants for the evaluation study were invited to participate on a voluntary 
basis from HEPE 129 students who enrolled in the course in the 1998 summer semester. 
Volunteers agreed to take four or five of the HEPE 129 lessons online. This was 
intended to give them experience with both methods of instruction. 

Assessment 



Students in the current summer course were required to take six exams in the 
testing center. Each exam covered material from two or three lessons in the course. The 
study focused on the seven lessons associated with exams three through five. All 
students took the same examinations in the testing center. 

Evaluation Groups 

Volunteer students were randomly divided into three groups. Two groups studied 
the content for a specific module using the online lesson material only. One of the three 
groups remained in the classroom to receive the classroom version of the instruction. 

This was done on a rotating basis for each of the three exams to counter any inequity in 
the groups unintentionally caused by the group selection technique. 

Survey Questionnaires 

A questionnaire was attached to the end of each exam associated with the study. 
Students responded to questions about how they prepared for the exam, their reaction to 
the instruction and whether they felt they had learned something from the lessons. 

Classroom Visits and Focus groups 

Classroom visits were made to collect attendance information and to make 
observations. Focus groups and interviews were conducted to obtain data from online 
and in-class students. 

Data Analysis 

Data was gathered from test results for both in-class and online students. Mean 
scores and standard deviations were calculated for each group. A measure of statistical 
significance was used to calculate the effect size. In addition, survey results were 
tabulated; classroom observations and interview information was organized by topic in 
order to identify issues and trends. 
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LIMITATIONS 



There were several restricting circumstances that affected how we conducted this 
study. Only two students were enrolled for the full online course due to the fact that the 
course had just been made available to students. The low enrollment meant we were not 
able to obtain adequate data from students taking the online course as the sole means of 
instruction. Because of this, this evaluation did not study a full implementation of the 
online course. This means that we were not able to make a comparison of course 
completion rates and of the overall online learning experience. 

The design of the study reflects the limitations we faced. Specifically, this study 
compares seven online lessons to their corresponding classroom lessons. As a result, 
some of the data we collected may not necessarily describe differences in the two 
delivery methods but rather it may describe the course in general. We did not use the 
online lesson exams, fitness assessment, the exercise log or the online final examination. 
Thus we could not assess the effectiveness of these aspects of the course. We were, 
however, able to make comparisons in student performance on specific lessons and 
student preferences for instructional methods. A study using the fully implemented 
online course is something that will have to take place when the numbers of students 
choosing that option increases. 

In addition, we make the assumption that the assessment instruments being used 
by classroom teachers are valid for this course. Other than an item analysis to determine 
the degree of difficulty for each item, we did not complete an evaluation of the testing 
instruments used. We would have preferred that both groups take both the classroom and 
online exams but the logistics of this were impractical. It is possible that using the 
classroom examinations may be a disadvantage or an advantage for one of the groups. 
Using only the one set of exams was justified in that students are given course credit for 
passing the classroom exams in their current state. These exams presumably represent 
what students need to know whether they took the instruction online or in the classroom. 



There were one hundred and thirteen students initially enrolled in the summer 
semester ofHEPE 129. Ninety-six students were in attendance when we solicited student 
participation in the study. Thirty-nine students volunteered to participate by taking 
online lessons. About fifteen student volunteers indicated they would be taking the 
lessons on their home computer. The remaining students had access to computer labs on 
campus. Table 1 shows the group sizes for each examination. Table 2 shows the break 
down of students in each group by year in school. 

All students taking examinations one, two and six were assigned to take the instruction in 
the classroom. Actual group numbers for exams three, four and five were determined 
from questionnaire results. Only those students who clearly indicated that they 
completed the relevant lessons exclusively online or in the classroom were used. Table 3 
gives the examination results by group for each of the six exams. 
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Table 1 



Group Sizes 



* 



Exam Period 




Number of Students 


Total 


Online 

¥ 11 — 

Assigned Actual 


In Class 


Start of the course 


113 


“ 


-- 


— 


First Exam 


110 


“ 


— 


— 


Second Exam 


109 


— 


-- 


“ 


Third Exam 


110 


25 


18 


85 


Fourth Exam 


107 


26 


16 


76 


Fifth Exam 


106 


27 


13 


74 


Sixth Exam 


104 


— 


— 


— 



based on number of students taking exams with the exception of the initial enrollment 
includes only those who clearly indicated that they completed the lessons exclusively online 
**’ includes only those who clearly indicated that they completed the lessons exclusively in class 



Table 2 

Group Demographics by Exam Taken 



Year in College 


Examination 




Online 






In Class 






1st 


2nd 


3rd 


4th 


1st 


2nd 


3rd 


4 th 


First 


— 


— 


— 


— 


58 


13 


5 


24 


Second 


— 


— 


— 


— 


58 


13 


5 


24 


Third 


35 ’ 


35 


6 


24 


64 


8 


6 


23 


Fourth 


33 


27 


7 


33 


66 


8 


5 


21 


Fifth 


57 


7 


14 


22 


60 


12 


4 


24 


Sixth 

— w — : 


— 


— 


— 


— 


58 


13 


4 


25 



Number represents the percent of all students taking the exam who were in their first year at BYU 



Table 3 

Mean and Standard Deviation of Tests by Group 



Exam 


All Students 


Online 


In-Class 


Mean* 


S.D. 


mean 


S.D. 


mean 


S.D. 


Body Mind Spirit & Substance Abuse 


82.7 


8.6 


— 


— 


— 


— 


Cardiovascular Health 


80.1 


14.3 


-- 


— 


— 


— 


Cardiorespiratory & Muscular Fitness 


82.4 


11.0 


79.6 


10.8 


82.4 


10.8 


First Aid/Emergency Care & Human Sexuality 


81.1 


10.5 


81.1 


8.8 


81.2 


10.8 


Immunity & Infection, Cancer & Nutrition 


83.1 


10.9 


82.9 


10.2 


83.6 


10.8 


Consumer Health & Stress Management 

— rr ... - 


71.5 


13.7 


— 


— 


— 


— 



based on highest retake mark 
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The effect size calculations shown in Table 4 provide a measure for comparing 
examination results of the online and in-class groups. The measurement indicates that the 
resulting means vary by 26% of one standard deviation for exam three, 1% of one 
standard deviation for exam four, and 17% of one standard deviation in exam five. 
According to Cohen's (1977) rule of thumb, the practical significance of these differences 
is low to moderate. This indicates that factors other than the method of instruction 
account for a large portion of the variability in the results (Toothaker, 1996). 

Table 4 

Measure of practical significance 



Assessment 


Effect size 


Exam Three 


.26 


Exam Four 


.01 


Exam Five 


.17 



At the end of each exam students were asked to indicate how well they prepared 
themselves to take the examination. This was meant to give an indication as to how fully 
students participated in the instruction provided. Students from each group were then 
divided into two subgroups - students who participated fully in the instruction and those 
who only partially made use of the instruction provided. We expected that students 
would do better if they actually used the instruction provided. If no difference was 
noticed then that method of instruction may be ineffective. 

Students who indicated frill in-class participation included those who said they 
attended every class and did some or all the readings. Students who participated frilly in 
the online experience included those who said they completed all the readings, self-check 
questions, Speedback tests, and looked at some additional links. Partial in-class 
participants included those who said they did some of the reading and attended some of 
the classes. Partial on-line participants included those who indicted that they did the 
readings only. Table 5 shows the achievement results by group. As expected, students 
who utilized the instruction more frilly achieved better results but only slightly better. The 
improvement in learning tended to be only one or two questions on any given exam. 

Table 5 

Mean Scores Based on Participation bv Group 



Assessment 


Full Participation 


Partial Participation 


Online 


In-class 


Online 


In-class 


Exam Three 


86 


84 


77 


77 


Exam Four 


83 


83 


76 


76 


Exam Five 


83 


85 


81 


79 
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Attendance was taken as part of the classroom observations. Attendance results 
are shown in Table 6. On average 107 students took each exam but only 87 students 
attended class or took the lessons online. From these observations we conclude that as 
many as 30 students failed to attend any given class but still took the exam. 

Table 6 



Classroom Attendance 



Class 


Start of 
Lecture 


0-5 min 
late 


5-10 min 
late 


10-30 min 
late 


30+ min 
late 


Total in 
attendance 


Left class 
early 


July 8 a 


93 






1 


2 


96 


2 


July 10 bc 


56 


5 








61 




July 20 bc 


59 


7 


3 


4 




73 




July 22 bc 


54 


6 


3 


1 




64 


1 


July 27 bc 


57 


11 


1 


1 




70 




July 31 bc 


42 


6 


16 


9 




73 




Aug 3 c 


47 


19 


8 


6 




80 




Aug 10 a 


66 


6 


5 


6 

I7TT _ 


4 


87 


16 



a pre/post fitness assessments handed out in this class Viewed first 30 min of class only c This 
class occurred during the study 



From the secretary of HEPE we learned that there is a general trend for class 
attendance. At the beginning of the semester attendance is the highest. Attendance drops 
off considerably in the middle of the course and increases again towards the end of the 
semester. This is consistent with our observations. In addition, we see that the overall 
classroom attendance did not increase very much after the students taking online lessons 
were scheduled to return to the classroom. 

After each exam students were asked to describe their learning experience (see 
Table 7) and whether they felt they had learned anything that had motivated them to 
change their lifestyle (see Table 8). The results were fairly consistent between groups. 
Of significance is the fact that about half of the students in both groups felt they learned 
many things in the lesson that they planned to use. Over 60% of both groups indicated 
they had learned something in the lesson that motivated them to change their lifestyle. 

Table 7 

Questionnaire Results for the Question Asking Students to Describe What They Learned. 



Response 


Online 


In-class 


Little I didn't already know 


15% 


7% 


Some but not significant 


10% 


12% 


Some but not relevant to me 


27% 


27% 


Lots, which I intend to use 


48% 


54% 
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Table 8 



Questionnaire Results for the Question Asking Students If They Had Learned Anything 
that Motivated a Change in Lifestyle. 



Response 


Online 


In-class 


Yes 


62% 


67% 


No 


38% 


33 % 



Based on an item analysis of the examination questions, we found that some 
questions tended to favor one group over the other. There were three versions of each 
exam for a total of 363 questions for exams three, four and five. For each question the 
degree of difficulty was calculated by group. There were 71 questions that favored one 
group or the other by a degree of difficulty of 0.2 or more. Of these, 23 questions 
favored the online group and 48 favored the in-class group. In fact, several students in 
both groups commented that the examinations tested content that the instruction didn't 
cover. Several classroom students felt they got mixed information from instructors and 
as a result didn't know what to answer on some questions. Many students using the online 
material felt that there was too much information to learn and that there should be more 
direction as to what they needed to study and what was going to be tested on the exam. 
Correspondingly, 52% of the online students felt the instruction didn't prepare them very 
well for the exams while 56% of the in-class students indicated that the classroom 
instruction prepared them about the same as most instruction does for exams. A 
summary of these results is shown in Table 9. 

Table 9 

Questionnaire Results for the Question Asking Students to Describe How Well the 
Instruction Prepared Them for the Exam. 



Response 


Online 


In-class 


Not very well 


52% 


19% 


About the same as most instruction 


33 % 


56% 


Slightly better than most instruction 


10% 


11 % 


Very well 


4% 


14% 



Of those students who chose not to take online lessons, 27% indicated that they 
preferred the classroom experience because they felt they would get important exam hints 
and helps in class that they wouldn't get online. Some instructors actually presented test 
questions and answers verbally in class. The fact that the online experience did not 
provide specific information regarding test preparation may have affected students' 
perceptions of how well the instruction prepared them for the exam. 
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Technology problems were identified as a possible concern early in the study. 
Students who took the online lessons were asked whether they experienced any problems 
with the technology. Table 10 shows the student responses. Several students indicated 
that they had problems with the videos from the CD-ROM disks provided. Most were 
unaware that there were four CD-ROM disks and had only obtained one of the four. 

Other problems included not having a sufficiently current Internet browser to access the 
lessons and using inadequate hardware. 

It should be noted that the classroom lecturers also experienced recurring 
problems with technology. Some of the problems included having difficulty with 
PowerPoint presentations, projector problems, and squealing or non-functioning 
microphones. Some of the problems were minor; others delayed or otherwise affected the 
presentation. 

Table 10 

Questionnaire Results for the Question Asking Students to Describe Problems with the 
Online Technology. 



Response 


Percent of Online Students 


It was a real problem 


17% 


I had some difficulty 


35% 


I had minor problems 


24% 


No problems 


24% 



Many of the online students indicated that, while taking the lessons, they had 
questions that they would like to have asked. Some of the questions were about the 
technology and others were about the course. A few who did ask questions via email 
indicated the technical responses they received from Independent Studies were helpful. 
When asked about this, classroom students said that they also had questions they would 
like to have asked. Some felt they shouldn't waste other students' time during class and 
would approach the instructor afterwards. Others felt that in a large class they had very 
little access to the instructor. 

After completing a group of online lessons, students were asked whether they 
would prefer to finish the course in the classroom or online. Table 1 1 gives a summary 
of the student responses. 
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Table 11 



Questionnaire Results for the Question Asking Students How They Would Prefer to 
Finish the Course. 



Response 


Percent of Online Students 


In the classroom 


57% 


Online 


16% 


Either would be fine 


27% 



CONCLUSION 



Student Performance 



Do students using different methods for delivering this instruction achieve similar 
results on assessment instruments currently being used in the on-campus course ? Based 
on the measures of practical significance, the test scores were not significantly different. 
This leads us to believe that either method for providing instruction could satisfactorily 
be used to learn the content of this course. 

Student Attitudes and Behavior Question. 

Do students feel their attitudes and behavior have changed after having taken this 
course ? About two thirds of the students indicated they had learned something that 
motivated them to make a lifestyle change. In general, there seemed to be a high level of 
enthusiasm for this course. Several students indicated that they found the course to be a 
lot better than they expected it would be. Most were optimistic that the changes they 
were making in their lifestyle would be permanent. This optimism was found in both 
groups but due to the limitations of the study's design, this optimistic attitude may speak 
more to the course in general and not to differences in instructional delivery methods. 

Value of the Instruction Questions. 

Do students use the instructional resources available ? Clearly not all students are 
accessing the available instruction. Classroom attendance was often low and many 
online students did little more than read the materials presented. This however did not 
always translate into poor grades. Even students who indicated that they did only 
minimal preparation for examinations tended to achieve adequate scores. The exam 
retake policy, previous knowledge and student's coping skills may be reasons for this. 
The results did tend to show that, with either experience, learning is more likely to take 
place when students participate fully. This would indicate that both methods of 
instruction could lead to adequate learning if fully experienced. 
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Do students prefer one delivery method more than the other ? Of those students 
who participated by taking online lessons, 73% had a definite opinion about which 
experience they preferred. Overall, 57% of the students indicated that they preferred the 
classroom experience. Some said they enjoyed the association of friends and classmates. 
Others indicated that they remember things they have heard better than things they have 
read. They also enjoyed the entertainment aspect of the live instruction. Several students 
indicated that they needed the structure of a regularly scheduled class in order to be 
successful at their studies. Others said that they didn't have easy access to a computer 
and that if they had to come on campus to study they might as well attend class. 

Most students who indicated that they preferred the online experience enjoyed the 
flexibility. Some couldn't commit to attending class at a set time for personal reasons. 
Others felt it would save them time doing the lessons online. 

While more than half the students who participated in the online experience 
indicated they would prefer to take subsequent lessons in the classroom, the students' 
reasons for choosing one method over another seemed to have little to do with the quality 
of the instruction. An individual's personal situation, social issues and the perceived 
benefits of the specific delivery method were often a determining factor in a student's 
preference. 

Do students find one method of instruction more useful than the other ? While 
students taking the lessons online tended to feel that the instruction didn't prepare them 
very well for the exam, their performance was not noticeably affected. Many students 
who indicated they preferred classroom instruction said that their choice was largely 
influenced by the fact that exam hints given in class better prepared them for the tests. 
Most students felt they could learn either way but preferred to be told specifically what 
they needed to know for the exams. It is clear that a part of the classroom experience 
included lecturers emphasizing content that would be covered on the exam. 



DISCUSSION 

Neither method of instruction provided for this course is perfect, and students 
found cause to criticize both. Both instructional systems did however do an acceptable 
job of meeting the objectives of the course. Students learned correct principles of health 
and physical fitness and they were expected to apply those principles in designing and 
following a personalized fitness and lifestyle management program. As an added bonus, 
students also tended to enjoy the course and felt that they were making positive lifestyle 
changes. 
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HEPE 129 Online 



Although some problems exist that need to be worked out, the online course for 
HEPE 129 is a well-designed course; not every online course is. As well as the required 
course information, this online course has a wealth of additional learning opportunities 
that unfortunately may go unused by the performing and conforming learners taking this 
course. Many students commented that there was just too much information to look at 
and that they skipped videos and Internet links that they felt wouldn't benefit them on the 
exam. Still the additional resources are there for those that wish to use them. 

In addition to the amount of information, students felt that they needed help with 
exam preparation. This may have occurred because this study did not utilize the entire 
online course. The online assessments are open book and the final assessment does have 
a "how to prepare for the exam" section. In addition, the objectives for each lesson are 
clearly provided. Still the design of the course might be improved by clearly identifying 
the principles of health and fitness that students are required to know for the exam. 

There were also some specific programming problems that will need to be 
corrected. Specifically, the Speedback tests for some lessons malfunctioned and did not 
allow students to submit their answers. The navigation buttons for some of the pages 
failed to move students to the appropriate pages. The labels on the CD-ROM disks need 
to be standardized so videos will work on Macintosh and Windows based computers. 
These problems will no doubt be corrected as field testing of the course is carried out. 

Time and Pace 

One of the greatest benefits of taking the course online is the flexibility of being 
able to work on a lesson when it is convenient. Students can control the time and pace of 
the instruction. This flexibility is an extremely important advantage for many students. 
One potentially problematic side effect of this flexibility is the likelihood that students 
will not find a convenient time to work on lessons. Course completion rates are 
something that this study was not able to track. However, based on research of distance 
learning and other computer assisted learning programs, this may be a problem. 

The classroom provides structure for some students and thus course completion 
rates using classroom instruction are often higher than those using online instruction. 
Students have been trained to attend class and use this as a means of organizing their 
daily schedule. The problem with this is that the learner has little or no ability to control 
the pace of the instruction or the amount of time they can spend with the instruction. 
Several students indicated that circumstances sometimes prevented them from attending 
class. In addition, some lectures used poorly designed slides that were ineffective or hard 
to read. Some students often fell asleep in class or were distracted by conditions in the 
classroom such as parents bringing young children to class. As one student put it "How 
do I make up for a class I miss?" One way might be to make both methods of instruction 
available to students. 
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From an instructional provider's point of view, we need to eliminate as many 
barriers to learning as we can. Ideally, students would be able to access those things they 
like about the classroom instruction and optionally have the flexibility of using online 
resources as needed. 

Human Interaction 



Another issue that needs to be addressed is that of communication. Discussion has 
been identified as an important aspect of learning (Collins, 1994), yet how do students get 
access to instructors in order to discuss course content? This isn't a problem unique to 
online courses but also exists in most large classroom situations. It would be impractical 
for one person to answer hundreds of students' e-mails every day. It is equally 
impossible for a professor to speak personally to two hundred students after class. 

Currently Independent Study does have a person who answers questions about the 
technology. It is likely that as more students enroll online this will not be enough. The 
online course has been designed as a self-study course and thus doesn't provide students 
with instructor access to answer their course content questions. 

The classroom course has a similar communication problem. Because of the 
structure of the course, many different professors teach each section. No one professor is 
responsible for a particular section. The HEPE secretary normally handles most student 
concerns but many students commented that they found it difficult to get information 
about the administrative details of the course. Many students noted that it was 
"comforting" to have the HEPE coordinator attend each class. This was possible in the 
summer session but will not be feasible in the winter and fall sessions when the number 
of sections increase from one to possibly five. Ideally class sizes would be kept small 
enough that an instructor could have personal contact with each student. This is not 
likely to happen as the number of students attending the University increases. 

Some potential solutions for both methods of instruction might be to maintain a 
site that provides answers to frequently asked questions. Many organizations promote a 
tutor system intended to provide a "human touch" to their distance learning materials. 

This is something that Independent Study may wish to consider. Another partial solution 
might be to create a moderated chat site for the course. The online version of this course 
does have a student bulletin board system available; however, none of the students 
accessed it during the study. Most students indicated that they would be happy to receive 
an e-mail response to their questions as long as it was within twenty-four hours of having 
asked the question. Any solution for this communication problem should take into 
account the need for students to discuss course content and get information about the 
course as well. 
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Recommendations 



As the number of students wishing to attend BYU increases, alternative means of 
instruction should be considered. It is clear that one method of instruction alone will not 
meet the needs of all students. The diverse needs of students warrant the existence of the 
classroom courses and equivalent online courses. While students tended to prefer the 
classroom experience, for some students, well-designed online instruction may better 
meet their needs. 

Whether technology is used to enhance classroom instruction or as the sole 
medium of instruction, there is a possibility that problems will occur. In our classroom 
courses and in distance learning courses, we need to continue to incorporate elements of 
instruction that promote learning. Our ability to provide effective online instruction will 
increase as technological advances are made and as we learn to better apply methods of 
instructional design. As quality online courses are developed, they will play an important 
part in the future of education. 
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